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Abstract 

In this paper we study a dynamic vehicle routing problem in which there are multiple vehicles 
and multiple classes of demands. Demands of each class arrive in the environment randomly 
over time and require a random amount of on-site service that is characteristic of the class. 
To service a demand, one of the vehicles must travel to the demand location and remain there 
for the required on-site service time. The quality of service provided to each class is given by 
the expected delay between the arrival of a demand in the class, and that demand's service 
completion. The goal is to design a routing policy for the service vehicles which minimizes 
a convex combination of the delays for each class. First, we provide a lower bound on the 
achievable values of the convex combination of delays. Then, we propose a novel routing policy 
and analyze its performance under heavy load conditions (i.e., when the fraction of time the 
service vehicles spend performing on-site service approaches one). The policy performs within 
a constant factor of the lower bound (and thus the optimal), where the constant depends only 
on the number of classes, and is independent of the number of vehicles, the arrival rates of 
demands, the on-site service times, and the convex combination coefficients. 



1 Introduction 

Consider a bounded environment E in the plane which contains n service vehicles. Demands for 
service arrive in E sequentially over time and each demand is a member of one of m classes. Upon 
arrival, a demand assumes a location in f , and requires a class dependent amount of on-site service 
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time. To service a demand, one of the n vehicles must travel to the demand location and perform 
the on-site service. If we specify a policy by which the vehicles serve demands, then the expected 
delay for demands of class a, denoted Da, is the expected amount of time between a demands 
arrival and its service completion. Then, given coefficients ci, . . . ,Cm > 0, the goal is to find the 
vehicle routing policy that minimizes 

CiDi H h CmDm- 

By increasing the coefficients for certain classes, a higher priority level can be given to their de- 
mands. This problem, which we call dynamic vehicle routing with priority classes, has important 
applications in areas such as UAV surveillance, where targets are given different priority levels 
based on their urgency or potential importance. 

In classical queuing theory (i.e., queuing systems in which the demands are not spatially dis- 
tributed), the problem of priority queues has received much attention, jTj. In p] the authors 
characterize the region of delays that are realizable by a single server. This analysis is performed 
under the assumption that the customer (demand) interarrival times and service times are dis- 
tributed exponentially. In [3] the achievable delays are studied in more a general setting known as 
queuing networks. 

If service demands are spatially distributed, then providing service becomes a problem in dy- 
namic vehicle routing (DVR) . One of the first DVR problems was the dynamic traveling repairper- 
son problem (DTRP) [H The DTRP is the single class version of the dynamic vehicle routing 
with priority classes problem studied in this paper. In |51 15] , the authors study the expected delay 
of demands and propose optimal policies in both heavy load (i.e., when the fraction of time the 
service vehicles spend performing on-site service approaches one), and in light load (i.e., when the 
fraction of time the service vehicles spends performing on-site service approaches zero) . In [7] , and 
[8], decentralized policies are developed for the DTRP. Spatial queuing problems have also been 
studied in the context of urban operations research [9] , where approximations are used to cast the 
problems in the traditional queuing framework. In our previous paper [10], we introduced and 
studied dynamic vehicle routing with priority classes, for the case of two classes and one vehicle. 
For this case we derived a lower bound on the achievable delay values and proposed the Random- 
ized Priority policy, which performed within a constant factor of the lower bound, for all convex 
combination coefficients. 

The contributions of this paper are as follows. We extend the dynamic vehicle routing with 
priority classes problem to n service vehicles and m classes of demands. The extension of our 
previous analysis to multiple classes of demands is very nontrivial. We derive a new lower bound 
on the achievable values of the convex combination of delays, and propose a new policy in which 
each class of demands is served separately from the others. We show that the policy performs with 
a constant factor of 2m? of the optimal. Thus, the constant factor is independent of the number 
of vehicles, the arrival rates of demands, the on-site service times, and the convex combination 
coefficients. We also comment on the source of the gap between the upper and lower bounds. 
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The paper is organized as follows. In Section [2] we give some asymptotic properties of the 
traveling salesperson tour. In Section 2.2 we formalize the problem and in Section |3] we derive a 
lower bound, and in Section [4] we introduce and analyze the Separate Queues policy. Finally, in 
Section [5] we present simulation results. 



2 Background and Problem Statement 

In this section we summarize the asymptotic properties of the Euclidean traveling salesperson tour, 
and formalize dynamic vehicle routing with priority classes. 



2.1 The Euclidean Traveling Salesperson Problem 

Given a set Q of N points in M^, the Euclidean traveling salesperson problem (TSP) is to find the 
minimum-length tour of Q (i.e., the shortest closed path through all points). Let TSP(Q) denote 
the minimum length of a tour through all the points in Q. Assume that the locations of the 
points are random variables independently and identically distributed, uniformly in a compact set 
£ with area \£\; in [11] it is shown that there exists a constant /?tsp such that, almost surely, 

hm 5^ = /3tspv1^. (1) 



The constant /3tsp has been estimated numerically as /9tsp ~ 0.7120 it 0.0002, [12 . The bound 
in equation ([T| holds for all compact sets £, and the shape of £ only affects the convergence rate 
to the limit. In [9j, the authors note that if £ is "fairly compact [square] and fairly convex", then 
equation ([T| provides an adequate estimate of the optimal TSP tour length for values of N as low 
as 15. 



2.2 Problem Statement 

Consider a compact environment £ in the plane with area \£\. The environment contains n vehicles, 
each with maximum speed v. Demands of type a £ {1, . . . , m} (also called a-demands) arrive in 
the environment according to a Poisson process with rate A^. Upon arrival, demands assume an 
independently and uniformly distributed location in £. An a-demand is serviced when the vehicle 
spends an on-site service time at the demand location, which is generally distributed with finite 
mean Sa- 

Consider the arrival of the ith a-demand. The service delay for the ith demand, Da{i), is the 
time elapsed between its arrival and its service completion. The wait time is defined as Wa{i) '■= 
Da{i) — ■Sq(z), where Sa{i) is the on-site service time required by demand i. A policy for routing 
the vehicles is said to be stable if the expected number of demands in the system for each class is 
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bounded uniformly at all times. A necessary condition for the existence of a stable policy is 

m 

i 



^ m 

-VA„s„<1. (2) 



n 



The load factor g is a standard quantity in queueing theory pj, and is used to capture the fraction 
of time the n servers (vehicles) must be busy in any stable policy. In general, it is difficult to study 
a queueing system for all values g £ [0, 1), and a common technique is to focus on the limiting 
regimes of ^ — > 1~, referred to as the heavy-load regime, and g 0"*", referred to as the light-load 
regime. 

Given a stable policy P the steady-state service delay for a-demands is defined as Da{P) ■= 
linij^+oo IE [Da{i)], and the steady-state wait time for a-demands is Wa{P) '■= Da{P) — Sa- Thus, 
for a stable policy P, the average delay per demand is 



^ m 



where A := X^oLi -^a- '^^^ average delay per demand is the standard cost functional for queueing 
systems with multiple classes of demands. Notice that we can write D{P) = ^^^i CaDa{P) with 
Ca = Aq/A. Thus, we can model priority among classes by allowing any convex combination of 
Di, . . . , Dm- If Ca > Aa/A, then the delay of a-demands is being weighted more heavily than in 
the average case. Thus, the quantity Cq,A/Aq, gives the priority of a-demands compared to that 
given in the average delay case. Without loss of generality we can assume that priority classes are 
labeled so that 

Cl Co Cm 

>/>•••> T^, (3) 
M ^2 Am 

implying that if a < /? for some a,/3 G {1, . . . ,m}, then the priority of a-demands is at least as 
high as that of /3-demands. With these definitions, we are now ready to state our problem. 

Problem Statement: Let 11 be the set of all causal, stable and stationary policies 
for dynamic vehicle routing with priority classes. Given the coefficients Ca > 0, a £ 
{1, . . . , m}, with Yl]^=i = li and satisfying equation Q, let D{P) := 'Y^=\ CaDa{P) 
be the cost of a policy P G 11. Then, the problem is to determine a vehicle routing 
policy P* , if one exists, such that 

D(P*) = inf D(P). (4) 

Pen 

We let D* denote the right-hand side of equation Q. A policy P for which D{P)/D* is bounded 
has a constant-factor guarantee. If limsup^.^^- D{P) / D* = k < -\-oo, then the policy P has a heavy- 
load constant-factor guarantee of k. In this paper we focus on the heavy-load regime, and look for 
policies with a heavy-load constant-factor guarantee that is independent of the number of vehicles, 
the arrival rates of demands, the on-site service times, and the convex combination coefficients. 
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3 Lower Bound in Heavy Load 



In this section we present two lower bounds on the delay in Eq. Q. The first holds only in heavy 
load (i.e., as g ^ 1~), while the second (less tight) bound holds for all g. 



Theorem 3.1 (Heavy load lower bound). In heavy load (g ^ 1 ), for every routing policy P, 

2n2?;2(l - q) 



a=l \ j=a+l 

where ci, . . . ,Cm satisfy Eq. 

Proof. Consider a tagged demand i of type a, and let us quantify its total service requirement. The 
demand requires on-site service time Sa(i). Let us denote by da{i) the distance from the location of 
the demand served prior to i, to z's location. In order to compute a lower bound on the wait time, 
we will allow "remote" servicing of some of the demands. For an a-demand i that can be serviced 
remotely, the travel distance da{i) is zero (i.e., a service vehicle can service the ith a-demand from 
any location by simply stopping for the on-site service time Sa{i))- Thus, the wait time for the 
modified remote servicing problem provides a lower bound on the wait time for the problem of 
interest. To formalize this idea, we introduce the variables G {0, 1} for each a S {1, . . . ,m}. If 
Tq = 0, then a-demands can be serviced remotely. If = 1, then a-demands must be serviced 
on location. We assume that Tq, = 1 for at least one a G {!,..., m}. Thus, the total service 
requirement of a-demand i is radaij) + Sa(i). The steady-state expected service requirement is 
r^da + Sa, where da ■= limj^^+oo IE [(ia(z)]. In order to maintain stability of the system we must 
require 



-f2Xa(— + -Se)<l. (6) 

Applying the definition of g in Eq. ([2]), we write Eq. Q as 

ra\ada < (1 " Q)'nv . (7) 



For a stable policy P, let represent the steady-state expected number of unserviced a- 
demands. Then, the expected total number of outstanding demands that require on-site service 
(i.e., cannot be serviced remotely) is given by X^JLi ''^j-^j- 1^°^ ^-PPly ^ result from the dynamic 
traveling repairperson problem (see jl3j . page 23) which states that in heavy load {g ^ 1~), if the 
steady-state number of outstanding demands is A^, then a lower bound on expected travel distance 
between demands is (/3tsp / V^) Applying this result we have that 
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for each a G {1, . . . ,m}. Combining with Eq. ([T]), squaring both sides, and rearranging we obtain 



< 



2 n^v\\ - qY 
From Little's law, = ^aWa for each a £ {1, . . . , m}, and thus 



^ raXaWa > 



2 n2t;2(l - gy 



(9) 



Recalling that Wa = Da — Sq and G {0, 1} for each a G {1, . . . , m}, we see that Eq. ([9| gives us 
2"* — 1 constraints on the feasible values of -Di(P), . . . ,Dm{P). Hence, a lower bound on D* can 
be found by minimizing Yl^=i subject to the constraints in Eq. By considering the dual of 
this problem, one can verify that under the class labeling in Eq. ([3]), the problem is equivalent to: 

m 

minimize CgWa, 



subject to 



Ai 





• 


• " 




'Wi' 
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• 
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A2 


As • 


Am_ 




Wm_ 




.(Ai + 


■ ■ + Am)^_ 



where 



2 n2t.2(l-^)2' 

Under the class labeling in Eq. ^ the above linear program is feasible and bounded, and its solution 
{Wl,...,W^) is given by 

(Q-l 

After rearranging, the optimal value of the cost function, and thus the lower bound on D*, is given 

by 



5^c„t^* = ^^ |c« + 2 Cj]Xc 

j=a+l 



a=l 



a=l 



Applying the definition of ^' we obtain the desired result. 



□ 
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Remark 3.2 (Lower bound for all g £ [0, 1)). With slight modifications, it it possible to obtain a 
less tight lower bound valid for all values of g. In the above derivation, the assumption that ^ — > 1~ 
is used in Eq. ([8|. It is possible to use, instead, a lower bound valid for all g E [0,1) (see J^): 



da > 1\ 



\£\ 



where 7 = 2/(3v27r) ~ 0.266. Using this bound we obtain the same linear program as in the proof 
of Theorem \3.1\ with the difference that ^ is now a function given by 

^'^\£\ n 
^^""^ nV(l-^>)2''" 2' 

Following the procedure in the proof of Theorem \3.1\ 



^1 = nV(l-^)2^^ " 2\i 



Q-l 



for each a G {2, . . . , m}. Finally, for every policy P , Da{P) > W* + Sa, and thus 



for all g G [0, 1) under the labeling in Eq. □ 

4 Separate Queues Policy 

In this section we introduce and analyze the Separate Queues (SQ) policy. We show that this policy 
is within a factor of 2m? of the lower bound in heavy load. 

To present the SQ policy we need some notation. We assume vehicle k S {1, . . . ,n} has a service 
region R^'^'^ C £, such that {.R^^', . . . , i?'""'} form a partition of the environment £. In general the 
partition could be time varying, but for the description of the SQ policy this will not be required. 
We assume that information on outstanding demands of type a G {1, . . . , m} in region at time 
t is summarized as a finite set of demand positions (^^'(i) with Na\t) := card(Qa^(t)) . Demands 
of type a with location in i?W are inserted in the set Qa^ as soon as they are generated. Removal 

\k] 

from the set Qa requires that service vehicle k moves to the demand location, and provides the 
on-site service. With this notation the policy is given as Algorithm 1. 
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Algorithm 1: Separate Queues (SQ) Policy 



Assumes: A probability distribution p = [pi, . . . ,Pm]- 

1 Partition £ into n equal area regions and assign one vehicle to each region. 

2 foreach vehicle-region pair k do 



if the set UaQa^ is empty then 



Move vehicle toward the median of its own region until a demand arrives. 



else 



Select Q G {Q^i \ • • • , Qm} according to p. 
if Q is empty then 
1^ Reselect until Q is nonempty. 

Compute TSP tour through all demands in Q. 

Service Q following the TSP tour, starting at the demand closest to the vehicle's 
current position. 
Repeat. 



12 Optimize over p. 



4.1 Stability Analysis of the SQ Policy in Heavy Load 

In this section we will analyze the SQ policy in heavy load, i.e., as ^ ^ 1~ . In the SQ policy each 
region i?!*^! has equal area, and contains a single vehicle. Thus, the n vehicle problem in a region 
of area l^"! has been turned into n independent single- vehicle problems, each in a region of area 
\£\/n, with arrival rates Xa/n. To determine the performance of the policy we need only study the 
performance in a single region k. For simplicity of notation we omit the label k. We refer to the 
time instant tj in which the vehicle computes a new TSP tour as the epoch i of the policy; we refer 
to the time interval between epoch i and epoch i -|- 1 as the ith iteration and we will refer to its 
length as Tj. Finally, let Na{ti) := N^^i, a S {1, . . . , m}, be the number of outstanding a-demands 
at beginning of iteration i. 

The following straightforward lemma, proved in [lOj, will be essential in deriving our main 
results. 

Lemma 4.1 (Number of outstanding demands). In heavy load (i.e., g after a transient, 

the number of demands serviced in a single tour of the vehicle in the SQ policy is very large with 
high probability (i.e., the number of demands tends to +oo with probability that tends to 1, as g 
approaches 1~ ). 

Let TSj be the event that Qj is selected for service at iteration i of the SQ policy. By the law 
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of total probability 



E[No,,^+l] =^PjE{N^,,+i\TSj), a G {!,..., m}, 
i=i 

where the conditioning is with respect to the task being performed during iteration i. During 
iteration i of the policy, demands arrive according to independent Poisson processes. Call N^^^ the 
a-demands (a G {1, . . . , m}) newly arrived during iteration i; then, by definition of the SQ policy 



if a = j 



E {NaATSj) + E N^T\TSj , o.w. 



E(Ar„,,+i|r5,- 



By the law of iterated expectation, we have K \^N^°^\TSjj = {Xa/n)K{Ti\TSj). Moreover, 
since the number of demands outstanding at the beginning of iteration i is independent of the task 
that will be chosen, we have E, {Na^i\TSj) = E [A'^a^j]. Thus we obtain 



E(iV,,,+i|TS,) 



'^E{T,\TSj), i(a = j 

E[N^,i] + ^E{Ti\TS,), o.w. 



Therefore, we are left with computing the conditional expected values of Tj. The length of Tj is 
given by the time needed by the vehicle to travel along the TSP tour plus the time spent to service 
demands. Assuming i large enough. Lemma (4.1 ) holds, and we can apply Eq. ([T]) to estimate from 
the quantities Na^i, a G {1, . . . , m}, the length of the TSP tour at iteration i. Conditioning on TSj 
(when only demands of type j are serviced) , we have 



E{T,\TSj) 



E(y]^|TS,)+E(ES^,,fc|T5, 



< ^^^^^ yii^ + E[iV,,].-,, 
where we have: (i) applied Eq. ([T|, (ii) applied Jensen's inequality for concave functions, in the 



form E y/X < y^E [X], (iii) removed the conditioning on TSj, since the random variables Na^i 
are independent from future events, and in particular from the choice of the task at iteration i, 
and (iv) used the crucial fact that the on-site service times are independent from the number of 
outstanding demands. 

Collecting the above results (and using the shorthand X to indicate E [X] , where X is any 
random variable), we have 



Na,i+1 <(1 -Pa)Na,i + '^Pj 

i=i 



A, 



(11) 
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for each a G { 1 , . . . , m} . The m inequahties above describe a system of recursive relations that 
allows to find an upper bound on Na,i, en £ {1, . . . ,m}. The following theorem (see Appendix for 
its proof) bounds the values to which they converge. 

Theorem 4.2 (Queue length). In heavy load, for every set of initial conditions {Nafl}ae{i,...,m}7 
the trajectories i i— > Na,i, a £ {1, ... , m}, resulting from Eqs. (11 ), satisfy 

1- AT ^ /^Tspl'^l -^a 
hmsupA'a^j < „ „r, — 

j^+oo n*V^(\ - qY Pa 

4.2 Delay of the SQ Pohcy in Heavy Load 

From Theorem |4.2[ and using Little's law, the delay of a-demands is 

Tl — 

Da{SQ) < — limsup A^'a^i + Sq, 




where we neglected Sa because of the heavy-load assumption. 

Thus, the delay (as defined in Eq. (|4])) of the SQ policy, satisfies in heavy load 

/?,t:L e^ . (12) 



71^1)2(1 — qY Pa 



With this expression we prove our main result on the performance of the SQ policy. 

Theorem 4.3 (SQ policy performance). In heavy load, the delay of the SQ policy is within a factor 
2m? of the optimal, independent of the arrival rates Ai, . . . , Xm, coefficients ci, . . . , Cm, service times 
si, . . . , Sm, and the number of vehicles n. 

Proof. We would like to compare the performance of this policy with the lower bound. To do this, 
consider setting Pa '■= Ca for each a S {1, . . . , m}. Defining B := /3^gp|<S|/(n^f ^(1 — qY), Eq. (12 ) 
can be written as 

D{SQ) < Em (X^x/^l • 



vi=l 



Next, the lower bound in Eq. ^ is 



i=l \ j=i+l I i=l 
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Thus, comparing the upper and lower bounds 



^ < o^ Jgi^)' (13) 



Letting xi := ^JciXi, and x := [xi, . . . the numerator of the fraction in Eq. (13) is ||x||^, and 

the denominator is ||x||2. But the one- and two-norms of a vector x G M™' satisfy ||x||i < -^77111x112. 
Thus, in heavy load we obtain 



D{SQ) 



< 2m (^jj-jp"^ ^ 



D* 

and the policy is a 2m^-factor approximation. □ 

Remark 4.4 (Relation to RP policy in [10 ). For m = 2 the SQ policy is within a factor of 8 of 
the optimal. This improves on the factor of 12 obtained for the Randomized Priority (RP) policy in 
fWj^ . However, it appears that the RP policy bound is not tight, since for two classes, simulations 
indicate it performs no worse than the SQ policy. □ 



5 Simulations and Discussion 

In this section we discuss, through the use of simulations, the performance of the SQ policy with the 
probability assignment pa ■= c^, for each a £ {1, . . . , m}. In particular, we study (i) the tightness 



of the upper bound in equation (12), (ii) conditions for which the gap between the lower bound 
in equation ^ and the upper bound in equation ( 12 ) is maximized, (iii) the suboptimality of the 
probability assignment p^ — Cq,, and (iv) the difference in performance between the SQ policy and 
a policy that merges all classes together irrespective of priorities. Simulations of the SQ policy were 
performed using linkerr{^as a solver to generate approximations to the optimal TSP tour. 

5.1 Tightness of the Upper Bound 

We consider one vehicle, four classes of demands, and several values of the load factor g. For 
each value of g we perform 100 runs. In each run we uniformly randomly generate arrival rates 
Ai, . . . , Am, convex combination coefficients ci, . . . , Cm, and on-site service times si, . . . ,Sm, and 
normalize the values such that the constraints Yl^=i ^aSa = Q and X]™=i Ca = 1 are satisfied. In 
each run we iterate the SQ policy 4000 times, and compute the steady-state expected delay by 
considering the number of demands in the last 1000 iterations. For each value of g we compute 
the ratio x between the expected delay and the theoretical upper bound in equation (12). Table [l] 



reports the ratio, its standard deviation, and its minimum and maximum values for each g value. 



'^The TSP solver linkern is freely available for academic research use at 



http : / /www . t sp . gat ech . edu/ Concorde . html 
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Figure 1: Experimental results for the SQ policy in worst-case conditions; g = 0.85 and Ai = 1. 



One can see that the upper bound provides a reasonable approximation for load factors as low as 
Q = 0.75. 



Load factor (q) 


EM 




maxx 


min X 


0.75 


0.803 


0.092 


1.093 


0.354 


0.8 


0.778 


0.108 


0.943 


0.256 


0.85 


0.773 


0.111 


1.150 


0.417 


0.9 


0.733 


0.159 


1.162 


0.203 


0.95 


0.716 


0.131 


0.890 


0.257 



Table 1: Ratio x between experimental results and upper bound for various values of g. 



5.2 Unfavorable Conditions for the SQ Policy 

One may question if for some sets {A^} and {cq,}, a G {1,..., m}, the ratio between upper bound 
(12) and lower bound ([s]) is indeed close to 2m^. The answer is affirmative: consider, e.g., the 



case Ai <C A2 ^ . . . Am and ci » C2 ^ . . . 3> Cm, with Aq,Cq = a, for some positive constant 
a. Then, the upper bound is equal to Brrfia and the lower bound is approximately equal to 
Bma/2, thus their ratio is (arbitrarily) close to 2m?. Then, we simulated the SQ policy for the 
case Am = a^m-i = a^Am-i = . . . = a™~^Ai and ci = ac2 = . . . = a"^~^Cm with a = 2. Fig. [l] 
shows that the experimental value of the cost function (averaged over 10 simulation runs) indeed 
increases proportionally to m?. 
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Figure 2: The ratios upbdj,/upbdopj for 2 classes of demands. 

5.3 Suboptimality of the Approximate Probability Assignment 

To prove Theorem |4.3| we used the probability assignment 

Pa '■= Ca for each a G {1, . . . , m}. (14) 



However, one would like to select [pi, . . . ,Pm] ='■ P that minimizes the right-hand side of Eq. (12|. 



The minimization of the right-hand side of Eq. (12) is a constrained multi- variable nonlinear op- 



timization problem over p, that is, in m dimensions. However, for two classes of demands the 
optimization is over a single variable pi, and it can be readily solved. A comparison of optimized 
upper bound, denoted upbd^p^, with the upper bound obtained using the probability assignment 
in Eq. (14), denoted upbd^, is shown in Fig. [2j 



For m > 2 we approximate the solution of the optimization problem as follows. For each 
value of m we perform 1000 runs. In each run we randomly generate Ai, . . . , Am, ci, . . . , Cm, and 
five sets of initial probability assignments pi,...,p5. From each initial probability assignment 
we use a line search to locally optimize the probability assignment. We take the ratio between 
upbd^ and the least upper bound upbdioj,a^i obtained from the five locally optimized probability 
assignments. We also record the maximum variation in the five locally optimized upper bounds. 
This is summarized in Table [2} The second column shows the largest ratio obtained over the 1000 



runs. The third column shows the largest % variation in the 1000 runs. The assignment in Eq. (14) 
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Number of classes (m) 



max upbd^/ upbd, 



^opt 



Max. % variation 



3 
4 
5 
6 
7 
8 



1.60 
1.51 
1.51 
1.74 
1.88 
1.63 



0.12 
0.04 
0.08 
0.02 
0.08 
0.15 



Table 2: Ratio of upper bound with p, 



•a 



Ca and upper bound with optimized p. 



performs within a factor of two of the optimized assignment. In addition, the optimization appears 
to converge to values close to a global optimum since all five random conditions converge to values 
that are within ~ 0.1% of each other on every run. 

5.4 The Merge Policy 

The simplest possible policy for our problem would be to ignore priorities and service demands all 
together, by repeatedly forming TSP tours of outstanding demands (i.e., by using the SQ policy as 
though there were only one class). We call such a policy the Merge policy. However, the performance 
of the SQ and the Merge policy can be arbitrarily far apart. Indeed, by defining the overall arrival 
rate A := 'Yl'^=i and overall mean on-site service S := ^1^=1 ^a-, and by using the upper bounds 
in [1], we immediately obtain as an upper bound for the Merge policy: D(Merge) < g-^i ■ 

Then, we see that D{Meige) / D{SQ) can be arbitrarily large by choosing Am ^ and Cm *C Cq, 
with a G {1, . . . ,m — 1}. This behavior is confirmed by experimental results, as depicted in Fig. 
|3] where we show the experimental ratios of delays between Merge and SQ policy (the ratios are 
averaged values over 10 simulation runs). 

6 Conclusions 

In this paper we studied a dynamic multi- vehicle routing problem with multiple classes of demands. 
For every set of coefficients, we determined a lower bound on the achievable convex combination 
of the class delays. We presented the Separate Queues (SQ) policy and showed that its deviation 
from the lower bound depends only on the number of the classes. We believe that there is room for 
improvement in the lower bound, and thus the SQ policy's performance may be significantly better 
than is indicated by its deviation from the current lower bound. Thus, our main thrust of future 
work will be in trying to raise the lower bound. We are also interested in combining the aspects 
of multi-class vehicle routing with problems in which demands require teams of vehicles for their 
service, and in extending our results to the case of non-uniform demand densities (possibly class 
dependent). 
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Figure 3: Ratio of experimental delays between Merge policy and SQ policy as a function of A2, 
with m = 2, Ai = 1, c = 0.995 and g = 0.9. 
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Appendix 



In this appendix we prove Theorem 4.2, Henceforth, we consider the relation "<" in as the 
product order of m copies of M (in other words, given two vectors v, w £ M™', the relation v < w is 
interpreted component- wise) . 



Proof of Theorem 4-^ Define qj := 1 — Pj and let Xa denote the arrival rate in region Thus 
Xa := Xa/n for each a £ {1, ... , m}. Let x{i) := (iVi^j, N2,i, • • • , -^m,i) S I^"* and define two matrices 



Xipisi+qi X1P2S2 
'X2P1S1 X2P2S2 + q2 

. XmPlSl XmP2S2 



and 



B :-- 



TSP 



nv 



XlPm^m 
X2PmSm 

XmPmSm ~l~ Qn 



XlPl X1P2 . . . XiPm 
X2PI X2P2 ■ ■ ■ X2Pm 

XmPl XmP2 ■ ■ ■ XmPm. 
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Then Eqs. (11 1 can be written as 



x{i + l) < Ax{i)+B 



_\/Xm{i). 



-■■ fix{i}) 



(15) 



where / : 



^>0 



^>o, and Xj{i), j G {1, . . . , m}, are the components of vector x{i). We refer to 



the discrete system in Eq. (15) as System-X. Next we define two auxihary systems, System-Y and 
System- Z. We define System-Y as 



y(i + l) = /(y(i)). 



(16) 



System-Y is, therefore, equal to System-X, with the exception that we replaced the inequality with 
an equality. 

Pick, now, any e > 0. From Young's inequality 



"v/a < - — h ea, for all a G M>o- 

4e 



(17) 



Hence, for i i-^ y{i) € M™q, the Eq. (16) becomes 



y{i + l) <Ay{i) + B(^j^lm + ey{i) 
= (^A + eB^y{i) + j^Blm. 



where Im is the vector (1,1,..., 1)""" E M™. Next, define System-Z as 



z(i + l) 



1 



A + £B]z(i) + Blr 



--■.g{z{{)). (18) 

The proof now proceeds as follows. First, we show that if x(0) = y(0) = z(0), then 

x{i) < y{i) < z{i), for ah i > (19) 



Second, we show that the trajectories of System-Z are bounded; this fact, together with Eq. (19), 
implies that also trajectories of System-Y and System-X are bounded. Third, and last, we will 
compute limsupj_,+oo y(0! this quantity, together with Eq. (19), will yield the desired result. 

Let us consider the first issue. We have y(l) = /(y(0)) and z{l) = g{z(0)). Since, by assumption 
z(0) = y(0), we have that ^(^(O)) = g{y{0)) > /(y(0)), where the last inequality follows from 
Eq. (17) and by definition of / and g . Therefore, we get y(l) < z(l). Then, we have y{2) = /(y(l)) 
and z{2) = g{z{l)). Since z(l),y(l) G M>q, and the elements in matrices A and B are all non- 
negative, then y(l) < z{l) implies g{y{l)) < g{z{l)). Using same arguments as before, we can write 
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-z(2) > g{y{l)) > /(y(l)) = x{2); therefore, we get y{2) < z{2). Then, it is immediate by induction 
that y(i) < z{i) for ah i >0. 

Similarly, we have x(l) < /(x(0)) = f{y{0)) = y(l), where we have used the assumption 
x(0) = y(0). Then, we get x(l) < y(l). Since x{l),y{l) S M>q, the elements in matrices A and 
B are nonnegative, and by the monotonicity of -^A, then x(l) < y(l) implies /(x(l)) < /(y(l)). 
Therefore, we can write x(2) < /(x(l)) < /(y(l)) = y(2); thus, we get x(2) < y(2). Then, it is 
immediate to show by induction that x{i) < y{i) for all i > 0, and Eq. ( |19| ) holds. 

We now turn our attention to the second issue, namely boundedness of trajectories for System- 
Z (in Eq. (18)). Notice that System-Z is a discrete-time linear system. The eigenvalues of A are 
characterized in the following lemma. 

Lemma 6.1. The eigenvalues of A are real and with magnitude strictly less than 1 (i.e., A is a 
stable matrix). 

Proof. Let w G C'" be an eigenvector of A, and G C be the corresponding eigenvalue. Then we 
have Aw = ^w. Define r := (piSi,p2S2; • • • ,PmSm)- Then the m eigenvalue equations are 

XjW-r + QjWj = fiWj, j £ {1, . . . , m}, (20) 

where w ■ r is the scalar product of vectors w and r, and wj is the jth component of w. 

There are two possible cases. If u; • r = 0, then Eq. (20) becomes qj Wj = fJ-Wj, for all j. Since 
w ^ 0, there exists j* such that w* ^ 0; thus, we have fi = qj*. Since qj* £ M and < qj* < 1, we 
have that fi is real and < 1. 

Assume, now, that w ■ r ^ 0. This implies that fj, ^ qj and Wj ^ for all j, thus we can write 
for all j 

'^^ w-r (21) 



fi-qj 



Therefore 



Therefore, (21) can be rewritten as 



Wj = ^ Wl. 

Ai - Qj 



m r 

y^j^ = i. (22) 



Eq. (22 ) implies that the eigenvalues are real. To see this, write n = a + ib, where i is the imaginary 
unit: then 

ErjXj V - rjXj[{a — qj) — ib] 

. , a + ib-qj ~ (a - qj)^ + b"^ 
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Thus Eq. (22) implies 



>0 



that is, 6 = 0. Eq. (22) also implies that the eigenvalues (that are real) have magnitude strictly 
less than 1. Indeed, assume, by contradiction, that > 1, then we would have fJ, — qj>l — qj>0 
(recall that the eigenvalues are real and < qj < 1) and we could write 



m t 



rn . rn 



Q<1, 



and we get a contradiction. Assume, again by contradiction, that /i < —1, then we would trivially 
get another contradiction YHjLi ^j-^j/Ca* ~ ^j) < 0' since fj, — qj < 0. □ 

Hence, A G M™^™ has eigenvalues strictly inside the unit disk, and since the eigenvalues of 
a matrix depend continuously on the matrix entries, there exists a sufficiently small e > such 
that the matrix A + eB has eigenvalues strictly inside the unit disk. Accordingly, each solution 
i I— > z(i) £ M>Q of System-Z converges exponentially fast to the unique equilibrium point 



Ir> 



A-eB 



-1 1 



-Blm. 



(23) 



are bounded. Thus 



Combining Eq. (19) with the previous statement, we see that the solutions i i— > x{i) and i i— > y{i) 

(24) 

Finally, we turn our attention to the third issue, namely the computation of y := lim supj^_|_oQ y{i) . 
Taking the lim sup of the left- and right-hand sides of Eq. (16), and noting that 



limsupx(i) < limsupy(i) < -|-oo. 

i— >+oo i— >+oo 



lim sup \/ ya{i) = /hmsupya(i) for a G {1, 2, . . . , m}, 
since is continuous and strictly monotone increasing on M>o, we obtain that 



(1 - Pa)ya + Xa ^Pj ( ^'^^^J'^W + 

i=i ^ ^ 



Rearranging we obtain 



PaVa 



\ ST /'/^TSPVl'f , ^ . _ 



nv 



(25) 
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Dividing PaVa by piyi we obtain 



Va = — yi- 

MPa 



Combining Eqs. (25) and (26), we obtain 



\/ 1 ^ I / ^ / 

.7=1 



Thus, recalling that Aq = Aq/ti, we obtain 



/3^Qp|iS| Ac 



n^t;2(l — Q^) p^ 



Noting that from Eq. (24), limsupj^ i Na,i < Va, obtain the desired result. 
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